强化学习算法在解决稀疏和延迟奖励的复杂分层任务时需要许多样本。对于此类复杂的任务，最近提出的方向舵使用奖励再分配来利用与完成子任务相关的Q功能中的步骤。但是，由于当前的探索策略无法在合理的时间内发现它们，因此通常只有很少有具有高回报的情节作为示范。在这项工作中，我们介绍了Align-rudder，该王牌利用了一个配置文件模型来进行奖励重新分布，该模型是从多个示范序列比对获得的。因此，Align-Rudder有效地采用了奖励再分配，从而大大改善了很少的演示学习。 Align-rudder在复杂的人工任务上的竞争者优于竞争对手，延迟的奖励和几乎没有示威的竞争者。在Minecraft获得Diamond的任务上，Align Rudder能够挖掘钻石，尽管不经常。代码可在https://github.com/ml-jku/align-rudder上找到。 YouTube：https：//youtu.be/ho-_8zul-uy

在终生学习中，代理人在整个生命中都在不重复的一生中学习，就像人类一样，在不断变化的环境中。因此，终身学习带来了许多研究问题，例如连续领域的转移，这导致了非平稳的奖励和环境动态。由于其连续的性质，这些非平稳性很难检测和应对。因此，需要探索策略和学习方法，这些方法能够跟踪稳定的领域变化并适应它们。我们提出反应性探索，以跟踪和反应终生增强学习中持续的域转移，并相应地更新策略。为此，我们进行实验以研究不同的勘探策略。我们从经验上表明，政策阶级家族的代表更适合终身学习，因为它们比Q学习更快地适应了分销的变化。因此，政策梯度方法从反应性探索中获利最大，并在终身学习中显示出良好的结果，并进行了持续的领域变化。我们的代码可在以下网址提供：https：//github.com/ml-jku/reactive-ecploration。

在部分可观察到的马尔可夫决策过程（POMDP）中，代理通常使用过去的表示来近似基础MDP。我们建议利用冷冻验证的语言变压器（PLT）进行病史表示和压缩，以提高样品效率。为了避免对变压器进行训练，我们引入了Frozenhopfield，该菲尔德自动将观察结果与预处理的令牌嵌入相关联。为了形成这些关联，现代的Hopfield网络存储了这些令牌嵌入，这些嵌入是通过查询获得的查询来检索的，这些嵌入者通过随机但固定的观察结果获得。我们的新方法Helm，启用了Actor-Critic网络体系结构，该架构包含用于历史记录表示的历史模块的审计语言变压器。由于不需要学习过去的代表，因此掌舵比竞争对手要高得多。在Miligrid和Procgen环境上，Helm掌舵取得了新的最新结果。我们的代码可在https://github.com/ml-jku/helm上找到。

在现实世界中，通过弱势政策影响环境可能是昂贵的或非常危险的，因此妨碍了现实世界的加强学习应用。离线强化学习（RL）可以从给定数据集中学习策略，而不与环境进行交互。但是，数据集是脱机RL算法的唯一信息源，并确定学习策略的性能。我们仍然缺乏关于数据集特征如何影响不同离线RL算法的研究。因此，我们对数据集特性如何实现离散动作环境的离线RL算法的性能的全面实证分析。数据集的特点是两个度量：（1）通过轨迹质量（TQ）测量的平均数据集返回和（2）由状态 - 动作覆盖（SACO）测量的覆盖范围。我们发现，禁止政策深度Q网家族的变体需要具有高SACO的数据集来表现良好。将学习策略朝向给定数据集的算法对具有高TQ或SACO的数据集进行了良好。对于具有高TQ的数据集，行为克隆优先级或类似于最好的离线RL算法。

Accurate determination of a small molecule candidate (ligand) binding pose in its target protein pocket is important for computer-aided drug discovery. Typical rigid-body docking methods ignore the pocket flexibility of protein, while the more accurate pose generation using molecular dynamics is hindered by slow protein dynamics. We develop a tiered tensor transform (3T) algorithm to rapidly generate diverse protein-ligand complex conformations for both pose and affinity estimation in drug screening, requiring neither machine learning training nor lengthy dynamics computation, while maintaining both coarse-grain-like coordinated protein dynamics and atomistic-level details of the complex pocket. The 3T conformation structures we generate are closer to experimental co-crystal structures than those generated by docking software, and more importantly achieve significantly higher accuracy in active ligand classification than traditional ensemble docking using hundreds of experimental protein conformations. 3T structure transformation is decoupled from the system physics, making future usage in other computational scientific domains possible.

Variational autoencoders model high-dimensional data by positing low-dimensional latent variables that are mapped through a flexible distribution parametrized by a neural network. Unfortunately, variational autoencoders often suffer from posterior collapse: the posterior of the latent variables is equal to its prior, rendering the variational autoencoder useless as a means to produce meaningful representations. Existing approaches to posterior collapse often attribute it to the use of neural networks or optimization issues due to variational approximation. In this paper, we consider posterior collapse as a problem of latent variable non-identifiability. We prove that the posterior collapses if and only if the latent variables are non-identifiable in the generative model. This fact implies that posterior collapse is not a phenomenon specific to the use of flexible distributions or approximate inference. Rather, it can occur in classical probabilistic models even with exact inference, which we also demonstrate. Based on these results, we propose a class of latent-identifiable variational autoencoders, deep generative models which enforce identifiability without sacrificing flexibility. This model class resolves the problem of latent variable non-identifiability by leveraging bijective Brenier maps and parameterizing them with input convex neural networks, without special variational inference objectives or optimization tricks. Across synthetic and real datasets, latent-identifiable variational autoencoders outperform existing methods in mitigating posterior collapse and providing meaningful representations of the data.

Differentiable Architecture Search (DARTS) has attracted considerable attention as a gradient-based Neural Architecture Search (NAS) method. Since the introduction of DARTS, there has been little work done on adapting the action space based on state-of-art architecture design principles for CNNs. In this work, we aim to address this gap by incrementally augmenting the DARTS search space with micro-design changes inspired by ConvNeXt and studying the trade-off between accuracy, evaluation layer count, and computational cost. To this end, we introduce the Pseudo-Inverted Bottleneck conv block intending to reduce the computational footprint of the inverted bottleneck block proposed in ConvNeXt. Our proposed architecture is much less sensitive to evaluation layer count and outperforms a DARTS network with similar size significantly, at layer counts as small as 2. Furthermore, with less layers, not only does it achieve higher accuracy with lower GMACs and parameter count, GradCAM comparisons show that our network is able to better detect distinctive features of target objects compared to DARTS.

Deep learning techniques with neural networks have been used effectively in computational fluid dynamics (CFD) to obtain solutions to nonlinear differential equations. This paper presents a physics-informed neural network (PINN) approach to solve the Blasius function. This method eliminates the process of changing the non-linear differential equation to an initial value problem. Also, it tackles the convergence issue arising in the conventional series solution. It is seen that this method produces results that are at par with the numerical and conventional methods. The solution is extended to the negative axis to show that PINNs capture the singularity of the function at $\eta=-5.69$

The Government of Kerala had increased the frequency of supply of free food kits owing to the pandemic, however, these items were static and not indicative of the personal preferences of the consumers. This paper conducts a comparative analysis of various clustering techniques on a scaled-down version of a real-world dataset obtained through a conjoint analysis-based survey. Clustering carried out by centroid-based methods such as k means is analyzed and the results are plotted along with SVD, and finally, a conclusion is reached as to which among the two is better. Once the clusters have been formulated, commodities are also decided upon for each cluster. Also, clustering is further enhanced by reassignment, based on a specific cluster loss threshold. Thus, the most efficacious clustering technique for designing a food kit tailored to the needs of individuals is finally obtained.

Learning efficient and interpretable policies has been a challenging task in reinforcement learning (RL), particularly in the visual RL setting with complex scenes. While neural networks have achieved competitive performance, the resulting policies are often over-parameterized black boxes that are difficult to interpret and deploy efficiently. More recent symbolic RL frameworks have shown that high-level domain-specific programming logic can be designed to handle both policy learning and symbolic planning. However, these approaches rely on coded primitives with little feature learning, and when applied to high-dimensional visual scenes, they can suffer from scalability issues and perform poorly when images have complex object interactions. To address these challenges, we propose \textit{Differentiable Symbolic Expression Search} (DiffSES), a novel symbolic learning approach that discovers discrete symbolic policies using partially differentiable optimization. By using object-level abstractions instead of raw pixel-level inputs, DiffSES is able to leverage the simplicity and scalability advantages of symbolic expressions, while also incorporating the strengths of neural networks for feature learning and optimization. Our experiments demonstrate that DiffSES is able to generate symbolic policies that are simpler and more and scalable than state-of-the-art symbolic RL methods, with a reduced amount of symbolic prior knowledge.